Entropy production along a stochastic trajectory and an integral fluctuation theorem 
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For stochastic non-equilibrium dynamics like a Langevin equation for a colloidal particle or a 
master equation for discrete states, entropy production along a single trajectory is studied. It 
involves both genuine particle entropy and entropy production in the surrounding medium. The 
integrated sum of both As to t is shown to obey a fluctuation theorem (exp[— Astot]) = 1 for arbitrary 
initial conditions and arbitrary time-dependent driving over a finite time interval. 
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Introduction. - Can the notions appearing in the first 
and second law of thermodynamics consistently be ap- 
plied to mesoscopic non-equilibrium processes like drag- 
ging a colloidal particle through a viscous fluid 0,0 SET? 
Concerning the first law, Sekimoto interpreted the terms 
in the standard overdamped Langevin equation as dy- 
namical increments for applied work, internal energy and 
dissipated heat For the second law and, in particu- 
lar, entropy, a proper formulation and interpretation is 
more subtle. Entropy might be considered as an ensem- 
ble property and therefore not to be applicable to a single 
trajectory. On the other hand, the so-called fluctuation 
theorem BSSIlEIilElElElElEEl quite 
generally relates the probability of entropy generating 
trajectories to those of entropy annihilating ones which 
requires obviously a definition of entropy on the level of a 
single trajectory. While for a colloidal particle immersed 
in a heat bath it is pretty clear what the entropy change 
of the bath is, it is less obvious whether or not one should 
assign an entropy to the particle itself as well. 

The purpose of this paper is to show that consequent 
ada ptio n of a previously introduced stochastic entropy 
[ill Il8l | to the trajectory of a colloidal particle together 
with the present original discussion of its equation of mo- 
tion yields a consistent interpretation of entropy produc- 
tion along a single stochastic trajectory. Moreover, it 
leads to a lucid and concise identification of boundary 
terms in fluctuation relations. In fact, we show for arbi- 
trary time-dependent driving that the total entropy pro- 
duction obeys an integral fluctuation theorem which is 
related to but different from Jarzynski's non-equilibrium 
work relation |19| . The present definition of entropy also 
implies that the known steady-state fluctuation theorem 
holds for finite times rather than in the lon g-ti me limit 
only as previously in stochastic dynamics |9|, [lOj . In a fi- 
nal step, this approach is generalized to arbitrary driven 
stochastic dynamics governed by a master equation with 
time-dependent rates. 

Entropy along a trajectory. - As a paradigm, we con- 
sider overdamped motion x(t) of a particle with mobility 
fi along a one-dimensional coordinate in the time-interval 
< t < t subject to a force 

F(x,l) = -d x V(x,l)+f(x > l). (1) 



This force can arise from a conservative potential V(x, I) 
and/or be applied to the particle directly as f(x, I). Both 
sources may be time-dependent through an external con- 
trol parameter A(t) varied according to some prescribed 
experimental protocol from 1(0) = Iq to l(t) = l t . The 
motion is governed by the Langevin equation 

x = fiF(x, I) + C, (2) 

with stochastic increments modelled as Gaussian white 
noise with (C( T )C( r ')) = 2DS(t — t') where D is the 
diffusion constant. In equilibrium, D and fi are related by 
the Einstein relation D = Tu, where T is the temperature 
of the surrounding medium. We assume this relation to 
persist even in a non-equilibrium situation. Throughout 
the paper we set Boltzmann's constant to unity such that 
entropy becomes dimensionless. 

For a definition of entropy along the trajectory, we 
consider first the corresponding Fokker-Planck equation 
for the probability p(x, r) to find the particle at x at time 
r as 

d T p(x,r) = -d x j{x,r) = ~d x (fiF(x,l) - Dd x p{x,r)) . 

(3) 

This partial differential equation must be augmented 
by a normalized initial distribution p(x,0) = Po(x). It 
will become crucial to distinguish the dynamical solution 
p(x, t) of this Fokker-Planck equation, which depends on 
this given initial condition, from the solution p s (x, I) for 
which the rhs of eq. vanishes at any fixed /. The 
latter corresponds either to a steady state for / ^ or 
to equilibrium for / = 0, respectively. 

The common definition of a non-equilibrium Gibbs en- 
tropy 

S(t) = — J dxpix, t) lnp(x, r) = (s(t)) (4) 

suggests to define a trajectory-dependent entropy for the 
particle (or "system") 

s(t) = -lnp(x(r),r) (5) 

where the probability p(x, r) obtained by solving the 
Fokker-Planck equation is evaluated along the stochas- 
tic trajectory x(t). Obviously, for any given trajectory 



2 



x(t), the entropy s(r) depends on the given initial data 
Po(x) and thus contains information on the whole ensem- 
ble. For an equilibrium Boltzmann distribution at fixed 
/, this definition assigns an entropy 



s(x) = (V(x,l)-F(l))/T, 



(6) 



with the free energy = —T J dx exp[— V(x, l)/T]. 
The definition Q has been used previously by Crooks for 
stochastic microscopically reversible dynamics [ill ] and 
by Qian for stochastic dynamics of macromolecules [lij . 
Neither work, however, discusses the equation of motion 
for this stochastic entropy. 

Entropy production. - The rate of change of the en- 
tropy © is given by 



s(r) 



drpjx, t) _ d x p(x, t) 
p( x , t ) | x(t) p(x,t) 1x(t) J 

drPjx, T) j(x,T) ^ 

P( X > T ) \x(t) D P( X ' T )\x(t) 



(7) 
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The first equality identifies the explicit and the im- 
plicit time-dependence. The second one uses the Fokker- 
Planck equation. The third term in the second line can 
be related to the rate of heat dissipation in the medium 



q(r) = F(x, l)x = Ts m (r) 



(8) 



where we identify the exchanged heat with an increase 
in entropy of the medium s m at temperature T = D/fi. 
Then (|7J can be written as a balance equation for the 
trajectory-dependent total entropy production 



StotO) = S m {T)+s(T) 



d T p(x, t) 



D P{ X i T )\x{t) 

(9) 

which is our first central result. The first term on the rhs 
signifies a change in p(x, t) which can be due to a time- 
dependent A(t) or, even at fixed I, due to relaxation from 
a non-stationary initial state po(x) ^ p s {x, Iq). 

Upon averaging, the total entropy production rate 
i>tot{T~) has to become positive as required by the sec- 
ond law. This ensemble average proceeds in two steps. 
First, we average over all trajectories which are at time 
r at a given x leading to 



(x\x,r) = j(x,r)/p(x,T). 



(10) 



Second, we average over all x with p(x, r) as 

Stot(T) = (Stot(T)) = 



dxj^->0, (11) 
Dp{x,T) 

where equality holds in equilibrium only. Averaging the 
increase in entropy of the medium along similar lines 
leads to 

S m (r) ee (s Ta (r)) = (F(x,r)x)/T (12) 
dxF{x,T)j(x,T)/T. (13) 



Hence upon averaging, the increase in entropy of the sys- 
tem itself becomes S(t) ee (s(t)} = Stot{i~) — S m (r). On 
the ensemble level, this balance equation for the averaged 
quantities has previously been derived directly from the 
ensemble definition 10} E3- The key point of our ap- 
proach is that we have defined entropy production (or 
annihilation) along a single stochastic trajectory split- 
ting it up into a medium part and a part of the particle 
(system). Beyond the conceptual advantage, this identi- 
fication facilitates a discussion of fluctuation theorems. 

Fluctuation theorem. - Fluctuation theorems derive 
from the behaviour of the weight of a trajectory under 
"time-reversal" which associates with each protocol A(t) 
a reversed one A(r) = l(t — r) and a reversed trajectory 
x(t) ee x(t — t). For a given initial value xq ee x(0) = 
x(t) ee it and final value xt = x(t) = i(0) = Jo, the ratio 
of probabilities of the forward path p[x(r)|a;o] and of the 
backward path p[x(r)|io] can easily be calculated in the 
ath integral representation of the Langevin equation as 



In 



p[x(t)\x ] 
P[x(t)\xq] 



[ F(x,t)x dr/T = As n 



(14) 



If this quantity is combined with arbitrary normal- 
ized distributions for initial and final value po^o) and 
Pi( x o) = Pi( x t): respectively, according to 

D r M , n i _ , p[x(t)\x ] po(x ) nc ., 
R[x{t), A(r);p ,Pi] = In _ r ., ... , (15) 



As 



p[x(t)\x ] pi(x ) 
Po(xq) 



In: 



Pl( x t) ' 

one easily derives the integral fluctuation relation ^ 



(16) 



(e R ) = pI x ( t )\ x o\ Poi x o)e R 

x(t),X 

= Pl x ( T )\ x o] Pi( x o) = 1- (17) 

x(t),x 

Here, the average is over both initial values drawn from 
the (in principle arbitrary) initial distribution po(xo) and 
trajectories x(t) determined by the noise history £( T )- 
Since the normalized distribution p\ [x] is arbitrary, there 
are, even for fixed po^), an infinity of choices for R which 
obey the constraint (|17fl and its implication (R) > 0. 

At least two choices of p\ have physical meaning in 
the present context. First, for p\(x t ) = p(x,t) which is 
the solution of the Fokker-Planck equation for the given 
initial distribution po^oX the definition JSJ implies that 
the last term in (|16f) becomes the entropy change of the 
particle As along the trajectory. Hence (|17|l implies the 
integral fluctuation theorem 



-As 



= 1 



(18) 



which is our second main result. This integral theorem 
for Astot is truely universal since it holds for any kind 
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of initial condition (not only for Pq(xq) = p s (x Q ,X )), 
any timc-dcpcndcncc of force and potential, with (for 
/ = 0) and without (for / ^ 0) detailed balance at fixed 
I, and any length of trajectory t without the need for 
waiting for relaxation. Crucial for this universality is our 
identification of the boundary term in (|16[) as the change 
in entropy of the particle. 

As a second important choice, for / = and po t i(x) = 
p s (x,l ,t) = exp[—V(x,lo tt ) - F(ln.t)], one recovers 
Jarzynski's relation (exp[— Wd/T]) = 1 [lj| since in this 



R = As m +[V(xt,h)-V(x ,lo)-H*t)+Hlo)}/T = w d /T 

(19) 

where Wd is the part of the work which is irreversibly lost 
as dissipated into the medium. The difference between 
the two choices for p\{x) is subtle but important. In 
the first case, the fluctuation theorem holds for the total 
entropy change along the trajectory evaluated at the very 
end of the protocol. For Jarzynski's relation, one takes 
the distribution corresponding to equilibrium at the final 
value of I. The difference arises from relaxation of the 
system towards the final equilibrium state at constant 
At which further increases the averaged entropy of the 
particle. In fact, p\{x) = p(x, t) is the one choice which 
leads to the smallest (R) among all possible pi(x). 

For a steady state at constant A and constant force 
/ like for motion along a ring with periodic boundary 
conditions, by choosing po{x) = pi(x) = p s (x) in (|16|l . 
one obtains the stronger fluctuation relation [111 Il4j 



p(-R)/p(R) = e 



(20) 



Since with the definition |jSJ the last term in (|16|) is again 
the change in entropy of the system As, the quantity R 
becomes the total entropy change As tot = As m + As. 
Hence, one recovers the fluctuation theorem for the total 
change in entropy as 



p(-As t ot)/p(Astot) = e 



-As to 



(21) 



even for a finite length t of the trajectories. In contrast, 
previous derivations of this genuine fluctuation theorem 
within stochastic dynamics 0, E3 hold in the long-time 
limit only since they implicitly ignore what we call As 
and consider only As m . Since the former is bounded for 
finite potentials, the latter will always win in the long 
run. 

Generalizations. - It is obvious that the present dis- 
cussion holds as well for systems with more than one de- 
gree of freedom obeying overdamped coupled Langevin 
equations. Rather than spelling out the notational de- 
tails, we will now discuss a more general stochastic dy- 
namics on a discrete set {n} of states. Again, we aim 
at a consistent definition of an entropy along a trajec- 
tory without having available any a priori notion of heat 
contrary to the colloidal case above which facilitated the 
identification of entropy production in the medium there. 



Let a transition between discrete states m and n oc- 
cur with a rate w mn (l), which depends on an externally 
controlled time-dependent parameter l(r). The master 
equation for the time-dependent probability p n {r) then 
reads 

IV run (l) Pn (r)). (22) 

For a solution, an initial distribution p n (0) must be spec- 
ified as well. As above, the system is driven externally 
from Iq to It according to a protocol 1(t). For any fixed /, 
there is a steady state p^(l) which may or may not obey 
detailed balance Pn(0 w nm(l) = Pm(l) w mn(l)- 

A stochastic trajectory n(r) starts at tiq and jumps 
at times Tj from nj to ri^ ending up at n t . As entropy 
along this trajectory, we define 



s(t) = -lnj>„ (r )(r) 



(23) 



where p n (T)( r ) is the solution p n (r) of the master equa- 
tion 1221) for a given initial distribution p n (0) taken along 
the specific trajectory n(r). As in the colloidal case, this 
entropy will depend on the chosen initial distribution. 

The entropy s(r) becomes time-dependent due to two 
sources. First, even if the system does not jump, p„( T )(r) 
can be time-dependent either for time-independent rates 
due to possible relaxation from a non-stationary initial 
state or, for time-dependent rates, due to the explicit 
time-dependence of p„( T )(r). Including the jumps, the 
change of system entropy reads 



s(r) = - 



drPnjr) (t) 



E 



S(t-T4 In — 2- 

Pn- 



(24) 



Here, and in the remainder, we suppress notationally the 
time-dependence of both p n (r) and the rates w nm (r) in 
the jump terms. We now split up the rhs into a total 
entropy production and one of the medium as follows 

i tot (r),-^^-E^-^ln^^ (25) 



3 3 3 



and 



UJ -\- - 

MT^-^r-^ln-^ 



W - + 

3 3 



(26) 



such that the balance StotC 7 ") = *( T ) + *mW holds. 

The rational behind the identification l|26(l for the in- 
crease in entropy of the medium becomes clear after av- 
eraging over many trajectories. For this average, we need 
the probability for a jump to occur at r = Tj from rij~ 
to nj + which is p~. ijj)w n - n + (r). Hence, one gets 

S m (r) = (s m (T)) = p n w nk ln— —, (27) 

, Wkn 
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Stot(r) = (stot(r)) = ^2p n w n k In 



n ,k 



PnW n k 
PkWkn 



(28) 



and 



S(t) = {s(t))= YVw nfc hi^ (29) 



such that the global balance S to t(T) = S m (T) + S(r) with 
Stot^) > is valid. By averaging our stochastic expres- 
sions, we thus recover and generalize established results 
for the non-equilibrium ensemble en tropy b alance avail- 
able so far for the steady state only [lCj, |2(J, |2l| . 

For the fluctuation theorems, the stochastic quantity R 
is derived from the probability P[n(T)|no] of a trajectory 
n (t) to occur under the protocol 1(t) and the probability 
P[ri(T) |rio] for the reversed trajectory n{r) = n(t — t) to 
occur under the reversed protocol 1(t) = l(t — r). With 
an arbitrary initial distribution p^ and an arbitrary final 
distribution p x n it becomes 



R[n(r),X(T);p° n y n ] = In J- 



P[n(T)\n ]pl 



= At 



-In 



o 

Pn t 



From the 
(exp[-i?]) 



p [n{r)\n Q ]pl B 

(30) 

infinity of possible fluctuation relations 
= 1, we choose two important ones. First, 



for p° = Pn(lo) and p x n = p n {t), the last term in (f5T7f> be- 
comes the increase in system entropy and R = Astot the 
total entropy change. Hence, we have again the integral 
theorem <|18[) . Second, the choice of p\ that corresponds 
to Jarzynski's relation in the colloidal case above is im- 
plied in the theorem derived in Ref. |22j . Finally, in 
a steady state for time-independent rates, by choosing 
Pn = Pn = Pni one nas the detailed version (|21fl for the 
total entropy change for any finite length of the trajec- 
tory as exemplified for a molecular motor or enzym in 
Ref. [13 . 

Summarizing perspective. - We have expressed the 
entropy production along a single stochastic trajectory 
as a sum of an entropy production of the system and of 
the medium both for a colloidal particle and for general 
stochastic dynamics obeying a master equation. The to- 
tal entropy production obeys an integral fluctuation the- 
orem for arbitrary time-dependent driving, for arbitrary 
initial conditions and any length of trajectories. This 
theorem and the Jarzynski relation are both shown to 
be special cases of an infinity of possible fluctuation rela- 
tions. With the present definition of entropy, the detailed 
fluctuation theorem valid in steady states for the total 



entropy production holds even for trajectories of finite 
length. 

The trajectory dependent entropy of the particle could 
be measured experimentally for a time-dependent proto- 
col by first recording over many trajectories the prob- 
ability distribution p(x,r) from which the entropy s(r) 
of each trajectory can be inferred. With such data, one 
could also test the new integral fluctuation theorem l|18|) 
and compare it to Jarzynski's relation for the same proto- 
col. It will be interesting to derive, both experimentally 
and theoretically, the probability distribution of these en- 
tropy changes and to see, e.g., whether they are Gaussian 
for slow driving as is the dissipated work appearing in 
Jarzynski's relation |2flj . 
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